`:top

`!Sequential quadratic programming`! (`!SQP`!) is an `F33f`_`[iterative method`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Iterative_method]`_`f for `F33f`_`[constrained nonlinear optimization`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Nonlinear_programming]`_`f, also known as Lagrange-Newton method. SQP methods are used on `F33f`_`[mathematical`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Mathematics]`_`f problems for which the `F33f`_`[objective function`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Objective_function]`_`f and the constraints are twice `F33f`_`[continuously differentiable`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Continuously_differentiable]`_`f, but not necessarily convex.

SQP methods solve a sequence of optimization subproblems, each of which optimizes a quadratic model of the objective subject to a linearization of the constraints. If the problem is unconstrained, then the method reduces to `F33f`_`[Newton's method`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Newton's_method]`_`f for finding a point where the gradient of the objective vanishes. If the problem has only equality constraints, then the method is equivalent to applying `F33f`_`[Newton's method`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Newton's_method]`_`f to the first-order optimality conditions, or `F33f`_`[Karush–Kuhn–Tucker conditions`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Karush–Kuhn–Tucker_conditions]`_`f, of the problem.

>>Contents

• `F0af`_`[Algorithm basics`#algorithm-basics]`_`f

• `F0af`_`[The equality constrained case`#the-equality-constrained-case]`_`f

• `F0af`_`[The inequality constrained case`#the-inequality-constrained-case]`_`f

• `F0af`_`[The SQP algorithm`#the-sqp-algorithm]`_`f

• `F0af`_`[Practical implementations`#practical-implementations]`_`f

• `F0af`_`[Alternative approaches`#alternative-approaches]`_`f

• `F0af`_`[Implementations`#implementations]`_`f

• `F0af`_`[See also`#see-also]`_`f

• `F0af`_`[Notes`#notes]`_`f

• `F0af`_`[References`#references]`_`f

• `F0af`_`[External links`#external-links]`_`f

-─

>>Algorithm basics

Consider a `F33f`_`[nonlinear programming`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Nonlinear_programming]`_`f problem of the form:

min x f ( x ) subject to h ( x ) ≥ ≥ 0 g ( x ) = 0. {\\displaystyle {\\begin{array}{rl}\\min \\limits _{x}&f(x)\\\\{\\mbox{subject to}}&h(x)\\geq 0\\\\&g(x)=0.\\end{array}}}

where x ∈ ∈ R n {\\displaystyle x\\in \\mathbb {R} ^{n}} , f : R n → → R {\\displaystyle f:\\mathbb {R} ^{n}\\rightarrow \\mathbb {R} } , h : R n → → R m I {\\displaystyle h:\\mathbb {R} ^{n}\\rightarrow \\mathbb {R} ^{m_{I}}} and g : R n → → R m E {\\displaystyle g:\\mathbb {R} ^{n}\\rightarrow \\mathbb {R} ^{m_{E}}} .

The `F33f`_`[Lagrangian`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Lagrange_multiplier]`_`f for this problem is`:cite-ref-1[`F5bf`_`[1`#cite-note-1]`_`f]

L ( x , λ λ , σ σ ) = f ( x ) + λ λ h ( x ) + σ σ g ( x ) , {\\displaystyle {\\mathcal {L}}(x,\\lambda ,\\sigma )=f(x)+\\lambda h(x)+\\sigma g(x),}

where λ λ {\\displaystyle \\lambda } and σ σ {\\displaystyle \\sigma } are `F33f`_`[Lagrange multipliers`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Lagrange_multipliers]`_`f.

>>>The equality constrained case

If the problem does not have inequality constraints (that is, m I = 0 {\\displaystyle m_{I}=0} ), the first-order optimality conditions (aka `F33f`_`[KKT conditions`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Karush–Kuhn–Tucker_conditions]`_`f) ∇ ∇ L ( x , σ σ ) = 0 {\\displaystyle \\nabla {\\mathcal {L}}(x,\\sigma )=0} are a set of nonlinear equations that may be iteratively solved with `F33f`_`[Newton's Method`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Newton's_method_in_optimization]`_`f. Newton's method linearizes the KKT conditions at the current iterate [ x k , σ σ k ] T {\\displaystyle \\left[x_{k},\\sigma _{k}\\right]^{T}} , which provides the following expression for the Newton step [ d x , d σ σ ] T {\\displaystyle \\left[d_{x},d_{\\sigma }\\right]^{T}} :

[ d x d σ σ ] = − − [ ∇ ∇ x x 2 L ( x k , σ σ k ) ] − − 1 ∇ ∇ x L ( x k , σ σ k ) = − − [ ∇ ∇ x x 2 L ( x k , σ σ k ) ∇ ∇ g ( x k , σ σ k ) ∇ ∇ g T ( x k , σ σ k ) 0 ] − − 1 [ ∇ ∇ f ( x k ) + σ σ k ∇ ∇ g ( x k ) g ( x k ) ] {\\displaystyle {\\begin{bmatrix}d_{x}\\\\d_{\\sigma }\\end{bmatrix}}=-[\\nabla _{xx}^{2}{\\mathcal {L}}(x_{k},\\sigma _{k})]^{-1}\\nabla _{x}{\\mathcal {L}}(x_{k},\\sigma _{k})=-{\\begin{bmatrix}\\nabla _{xx}^{2}{\\mathcal {L}}(x_{k},\\sigma _{k})&\\nabla g(x_{k},\\sigma _{k})\\\\\\nabla g^{T}(x_{k},\\sigma _{k})&0\\end{bmatrix}}^{-1}{\\begin{bmatrix}\\nabla f(x_{k})+\\sigma _{k}\\nabla g(x_{k})\\\\g(x_{k})\\end{bmatrix}}} ,

where ∇ ∇ x x 2 L ( x k , σ σ k ) {\\displaystyle \\nabla _{xx}^{2}{\\mathcal {L}}(x_{k},\\sigma _{k})} denotes the `F33f`_`[Hessian matrix`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Hessian_matrix]`_`f of the Lagrangian, and d x {\\displaystyle d_{x}} and d σ σ {\\displaystyle d_{\\sigma }} are the primal and dual displacements, respectively. Note that the Lagrangian Hessian is not explicitly inverted and a linear system is solved instead.

When the Lagrangian Hessian ∇ ∇ 2 L ( x k , σ σ k ) {\\displaystyle \\nabla ^{2}{\\mathcal {L}}(x_{k},\\sigma _{k})} is not `F33f`_`[positive definite`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Definite_matrix]`_`f, the Newton step may not exist or it may characterize a stationary point that is not a local minimum (but rather, a local maximum or a saddle point). In this case, the Lagrangian Hessian must be regularized, for example one can add a multiple of the identity to it such that the resulting matrix is positive definite.

An alternative view for obtaining the primal-dual displacements is to construct and solve a local quadratic model of the original problem at the current iterate:

min d x f ( x k ) + ∇ ∇ f ( x k ) T d x + 1 2 d x T ∇ ∇ x x 2 L ( x k , σ σ k ) d x s . t . g ( x k ) + ∇ ∇ g ( x k ) T d x = 0. {\\displaystyle {\\begin{array}{rl}\\min \\limits _{d_{x}}&f(x_{k})+\\nabla f(x_{k})^{T}d_{x}+{\\frac {1}{2}}d_{x}^{T}\\nabla _{xx}^{2}{\\mathcal {L}}(x_{k},\\sigma _{k})d_{x}\\\\\\mathrm {s.t.} &g(x_{k})+\\nabla g(x_{k})^{T}d_{x}=0.\\end{array}}}

The optimality conditions of this `F33f`_`[quadratic problem`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Quadratic_programming]`_`f correspond to the linearized KKT conditions of the original problem. Note that the term f ( x k ) {\\displaystyle f(x_{k})} in the expression above may be left out, since it is constant under the min d {\\displaystyle \\min \\limits _{d}} operator.

>>>The inequality constrained case

In the presence of inequality constraints ( m I > 0 {\\displaystyle m_{I}>0} ), we can naturally extend the definition of the local quadratic model introduced in the previous section:

min d f ( x k ) + ∇ ∇ f ( x k ) T d + 1 2 d T ∇ ∇ x x 2 L ( x k , λ λ k , σ σ k ) d s . t . h ( x k ) + ∇ ∇ h ( x k ) T d ≥ ≥ 0 g ( x k ) + ∇ ∇ g ( x k ) T d = 0. {\\displaystyle {\\begin{array}{rl}\\min \\limits _{d}&f(x_{k})+\\nabla f(x_{k})^{T}d+{\\frac {1}{2}}d^{T}\\nabla _{xx}^{2}{\\mathcal {L}}(x_{k},\\lambda _{k},\\sigma _{k})d\\\\\\mathrm {s.t.} &h(x_{k})+\\nabla h(x_{k})^{T}d\\geq 0\\\\&g(x_{k})+\\nabla g(x_{k})^{T}d=0.\\end{array}}}

>>>The SQP algorithm

The SQP algorithm starts from the initial iterate ( x 0 , λ λ 0 , σ σ 0 ) {\\displaystyle (x_{0},\\lambda _{0},\\sigma _{0})} . At each iteration, the QP subproblem is built and solved; the resulting Newton step direction [ d x , d λ λ , d σ σ ] T {\\displaystyle [d_{x},d_{\\lambda },d_{\\sigma }]^{T}} is used to update current iterate:

[ x k + 1 , λ λ k + 1 , σ σ k + 1 ] T = [ x k , λ λ k , σ σ k ] T + [ d x , d λ λ , d σ σ ] T . {\\displaystyle \\left[x_{k+1},\\lambda _{k+1},\\sigma _{k+1}\\right]^{T}=\\left[x_{k},\\lambda _{k},\\sigma _{k}\\right]^{T}+[d_{x},d_{\\lambda },d_{\\sigma }]^{T}.}

This process is repeated for k = 0 , 1 , 2 , … … {\\displaystyle k=0,1,2,\\ldots } until some convergence criterion is satisfied.

>>Practical implementations

Practical implementations of the SQP algorithm are significantly more complex than its basic version above. To adapt SQP for real-world applications, the following challenges must be addressed:

• The possibility of an infeasible QP subproblem.

• QP subproblem yielding a bad step: one that either fails to reduce the target or increases constraints violation.

• Breakdown of iterations due to significant deviation of the target/constraints from their quadratic/linear models.

To overcome these challenges, various strategies are typically employed:

• Use of merit functions, which assess progress towards a constrained solution, non-monotonic steps or filter methods.

• Trust region or line search methods to manage deviations between the quadratic model and the actual target.

• Special feasibility restoration phases to handle infeasible subproblems, or the use of L1-penalized subproblems to gradually decrease infeasibility

These strategies can be combined in numerous ways, resulting in a diverse range of SQP methods.

>>Alternative approaches

• `F33f`_`[Sequential linear programming`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Sequential_linear_programming]`_`f

• `F33f`_`[Sequential linear-quadratic programming`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Sequential_linear-quadratic_programming]`_`f

• `F33f`_`[Augmented Lagrangian method`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Augmented_Lagrangian_method]`_`f

>>Implementations

SQP methods have been implemented in well known numerical environments such as `F33f`_`[MATLAB`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=MATLAB]`_`f and `F33f`_`[GNU Octave`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=GNU_Octave]`_`f. There also exist numerous software libraries, including open source:

• `F33f`_`[SciPy`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=SciPy]`_`f (de facto standard for scientific Python) has scipy.optimize.minimize(method='SLSQP') solver.

• NLopt (C/C++ implementation, with numerous interfaces including Julia, Python, R, MATLAB/Octave), implemented by Dieter Kraft as part of a package for optimal control, and modified by S. G. Johnson.`:cite-ref-kraft-2-0[`F5bf`_`[2`#cite-note-kraft-2]`_`f]`:cite-ref-3[`F5bf`_`[3`#cite-note-3]`_`f]

• `F33f`_`[ALGLIB`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=ALGLIB]`_`f SQP solver (C++, C#, Java, Python API)

• acados (C with interfaces to Python, MATLAB, Simulink, Octave) implements a SQP method tailored to the problem structure arising in `F33f`_`[optimal control`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Optimal_control]`_`f, but tackles also general nonlinear programs.

and commercial

• `F33f`_`[LabVIEW`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=LabVIEW]`_`f

• `F33f`_`[KNITRO`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=KNITRO]`_`f`:cite-ref-4[`F5bf`_`[4`#cite-note-4]`_`f] (C, C++, C#, Java, Python, Julia, Fortran)

• `F33f`_`[NPSOL`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=NPSOL]`_`f (Fortran)

• `F33f`_`[SNOPT`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=SNOPT]`_`f (Fortran)

• `F33f`_`[NLPQL`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=NLPQL]`_`f (Fortran)

• `F33f`_`[MATLAB`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=MATLAB]`_`f

• `F33f`_`[SuanShu`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=SuanShu_numerical_library]`_`f (Java)

>>See also

• `F33f`_`[Newton's method`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Newton's_method]`_`f

• `F33f`_`[Secant method`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Secant_method]`_`f

• `F33f`_`[Model Predictive Control`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Model_predictive_control]`_`f

>>Notes

`:cite-note-1`!1.`! `F0af`_`[↑`#cite-ref-1]`_`f `:citerefjorge-nocedal-and-stephen-j-wright2006`aJorge Nocedal and Stephen J. Wright (2006). `*Numerical Optimization`*. Springer. `F33f`_`[ISBN`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=ISBN_(identifier)]`_`f 978-0-387-30303-1.

`:cite-note-kraft-2`!2.`! `F0af`_`[↑`#cite-ref-kraft-2-0]`_`f `:citerefkraft1994`aKraft, Dieter (Sep 1994). "Algorithm 733: TOMP–Fortran modules for optimal control calculations". `*ACM Transactions on Mathematical Software`*. `!20`! (3): 262–281. `F33f`_`[CiteSeerX`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=CiteSeerX_(identifier)]`_`f 10.1.1.512.2567. `F33f`_`[doi`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Doi_(identifier)]`_`f:10.1145/192115.192124. `F33f`_`[S2CID`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=S2CID_(identifier)]`_`f 16077051. Retrieved 1 February 2019.

`:cite-note-3`!3.`! `F0af`_`[↑`#cite-ref-3]`_`f "NLopt Algorithms: SLSQP". `*Read the Docs`*. July 1988. Retrieved 1 February 2019.

`:cite-note-4`!4.`! `F0af`_`[↑`#cite-ref-4]`_`f KNITRO User Guide: Algorithms

>>References

• `:citerefbonnansgilbertlemar-chalsagastiz-bal2006`aBonnans, J. Frédéric; Gilbert, J. Charles; `F33f`_`[Lemaréchal, Claude`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Claude_Lemaréchal]`_`f; `F33f`_`[Sagastizábal, Claudia A.`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Claudia_Sagastizábal]`_`f (2006). `*Numerical optimization: Theoretical and practical aspects`*. Universitext (Second revised ed. of translation of 1997 French ed.). Berlin: Springer-Verlag. pp. xiv+490. `F33f`_`[doi`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Doi_(identifier)]`_`f:10.1007/978-3-540-35447-5. `F33f`_`[ISBN`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=ISBN_(identifier)]`_`f 978-3-540-35445-1. `F33f`_`[MR`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=MR_(identifier)]`_`f 2265882.

• `:citerefjorge-nocedal-and-stephen-j-wright2006`aJorge Nocedal and Stephen J. Wright (2006). `*Numerical Optimization`*. Springer. `F33f`_`[ISBN`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=ISBN_(identifier)]`_`f 978-0-387-30303-1.

>>External links

• Sequential Quadratic Programming at NEOS guide

`c`F0af`_`[↑ Back to top`#top]`_`f`a